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ABSTRACT 

Institutional report cards are increasingly being used by higher educational institutions to present 
academic outcomes to external audiences of prospective students and parents, as well as 
program and institutional evaluators. While some prospective students are served by national 
transparency measures most users mine information from the institutional web site, so even 
institutions at the bottom of the comparison spectrum need to manage their information 
dissemination to fulfill audience needs. An analysis of the Web-published institutional report cards 
and transparency reports of a representative sample, stratified by institutional type and 
geographic location, of 82 American higher educational institutions yielded results which 
influenced the development of a taxonomy for institutional report cards for higher educational 
institutions. 98% of institutions offered some type of transparency or data reporting related to 
student learning, with 89.3% offering some type of institutional report card, performance 
indicators, score card, dashboard, or student achievement reporting for consumers. At an 
average of 8.12 documents, state colleges and universities averaged the highest number of 
documents and webpages devoted to transparency reporting. Carnegie-class research 
institutions and state colleges and universities most consistently deployed a single office, typically 
named as an Office of Institutional Research to disseminate findings 64.7% of the time. 
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INTRODUCTION 

Institutional report cards are increasingly being used by higher educational institutions to present 
academic outcomes to external audiences of prospective students and parents, as well as 
program and institutional evaluators. While systems such as the Voluntary System of 
Accountability (VSA), the Integrated Postsecondary Education Data System (IPEDS), and 
University and College Accountability Network (U-CAN) have reached critical mass participation 
levels in order to provide comparative information for stakeholders in a standardized format, 
criticisms continue about the voluntary participation of its membership and lack of information 
regarding student experiences, student future plans, and learning outcomes. In addition, 
incomplete data sets and aggregated or collapsed data points as well as the lack of student 
progress or value-added features such as score percentiles, rankings, and recommendations, 
restricts system value for consumers. While some prospective students are served by national 
transparency measures such as VSA's College Portrait® and IPEDS' College Navigator®, most 
users mine information from an institutional web site, so even institutions at the bottom of the 
comparison spectrum need to manage their information dissemination to fulfill audience needs. 
An analysis of the Web-published institutional report cards and transparency reports of a 
representative sample, stratified by institutional type and geographic location, of 82 American 
higher educational institutions yielded results which influenced the development of a taxonomy of 
institutional report cards for higher educational institutions. 

The growth in the deployment of institutional report cards, confirmed by this study, seems to 
parallel recent legislation of the Every Student Success Act (P.L. 114-95, 2015), including reform 
to a more state-wide centered transparency initiative in which each state develops its own 
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standards. Institutional report cards may offer consumers the strongest evidence that a particular 
institution is effective in addressing its growth and community impact; its reflection of the 
demographic characteristics of the college district; and that measures taken to improve student 
success, instructional quality, and satisfaction rates have been effective. With data from these 
report cards increasingly being reported to state workforce investment boards and department of 
education structures for inclusion in performance evaluation funding awards, evidence is 
increasing that taxpayers and lawmakers are increasingly making value judgements about the 
educational quality reflected in the data. 

The current study emerges from a need to better quantify and qualify what good learning and 
instructional settings mean for consumers of higher educational products. Pressures from various 
stakeholder voices, including taxpayers, accreditation agencies, and student participants 
increasingly taking on proactive change in a spectrum from legislative action to student protest. 
Therefore, the need for transparency and meaningfulness of system data, particularly when 
linked to funding, has never been greater, yet literature in this field is not well-developed, perhaps 
because accountability stakeholders are tied to political processes. Increasingly, literature is 
recognizing that educational accountability, performance and support requires a coordinated 
effort of key stakeholder groups: state departments of education to collect data and evaluate 
performance of districts to form state benchmarks; accrediting bodies, for setting accreditation 
standards and directing local school boards regarding implementation plans; state boards for 
accrediting schools and overseeing academic programs; and institution leaders who create, 
adopt, implement, and evaluate plans. 


LITERATURE REVIEW 

The use of performance reporting has been linked to positive association and trust (Grosso & Van 
Ryzin 2011), allows trend performance to be analyzed (Gajewski, Mahnken & Dunton 2008), and 
is theorized to reduce corruption if accountability measures and electoral mechanisms are in 
place (Schatz 2013). With field research recognizing the alignment of management frameworks to 
performance reports such as budgets, expenses, annual reports, performance reports, and 
performance benchmarking, there is a need for institutions of higher education to move from 
static data systems to value-added systems that analyze data in light of dashboards, outcomes, 
or stated goals. In fact, Offenstein, Moore, and Shulock (2010) recommended the use of 
milestones and on-track indicators to monitor degree completion and serve as a decision making 
tool for designing interventions. However, report card systems have their critics; in a report on the 
use of educational accountability systems in the United States, Means, Padilla, and Gallagher 
(2010) noted that less than half of the school districts can combine data from different systems 
and report on outcomes; while higher educational management systems are increasingly 
integrating campus databases, the wealth of database vendors in higher educational settings is 
indicative of similar integration issues. Due to stakeholder confusion of the dual state and federal 
reporting systems, South Carolina revised it performance reporting system to better include 
stakeholder input (South Carolina Legislative Services Agency 2015). Even a report sponsored 
by the National Center for Public Policy in Higher Education noted the lack of generalizability 
when comparing regional data mined from report cards (Swail, Jaeschke & Rasmussen 2009). 
More concerning is that research into the effectiveness of adopting performance-based 
measures to improve institutional performance is mixed, with Martinez and Nilson (2006) finding 
positive correlations and Shin (2009), Volkwein and Tandberg (2008) finding no relationship. 
While Hillman, Tanberg, and Gross (2014) found modest gains in a performance funding model 
compared to neighboring states, a systematic comparison found that that the performance 
funding model was ineffective for increasing college completions. In fact, Jacobsen, Snyder, and 
Saultz (2014) concluded that the effectiveness of report cards is mixed with greater transparency 
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either contributing to or diminishing legitimacy and support with format being a considerable 
factor in perception. 

Report Cards 

The use of a national report card system has seen an ebb and flow in American higher education, 
from a high-water mark in 2008 with the mandating of state Measuring Up report cards to the 
current system of annual reports with a focus on institutional reporting (Ginder, Kelly-Reid & 
Mann 2015). While all states are employing accountability systems, with annual reports as de- 
facto report cards; in general, while the data is comprehensive, it is not provided in an aggregated 
format conducive for providing a quick summary and rankings as a report card (Djurovich, et al. 
2015; New Mexico Higher Education Department 2013; Oklahoma State Regents of Higher 
Education 2015; State Council of Higher Education for Virginia 2015; State of Connecticut Board 
of Regents for Higher Education 2015; State of New Jersey Office of Research and Accountability 
Student Unit Record 2015; Texas Higher Education Accountability System 2015; West Virginia 
Higher Education Policy Commission and Community and Technical College System of West 
Virginia 2014; Wyoming Community College Commission 2010). Even states presenting a Ouick 
Facts or Fasts at a Glance PDF or web page are still formatting data in aggregate groups with an 
average of 48 facts per publication in this literature review sample (Colorado Commission on 
Higher Education 2015; Florida College System Research & Analytics 2015; Kentucky Council on 
Postsecondary Education 2015; Mississippi Institutions of Higher Learning, Office of Strategic 
Data Management 2015). While the Education Trust of the National Association of System Heads 
(NASH) provides benchmark reports of 22 American higher education systems as part of its 
Access to Success Initiative (Engle, Yeado, Brusi & Cruz 2012), data for non-participating states 
must be viewed in isolation. Independent projects such as the Young Invincibles' State Report 
Cards (2015) have stepped in to fill the void, however, the reliability of privately-sponsored 
datasets is a barrier for decision-making. 

Performance Metrics 

In order to determine how to classify groups of indicators for the coding form employed in this 
study, a representative sample of state and national performance reporting frameworks for higher 
education were explored for consideration in this study (Alabama Commission on Higher 
Education 2015; California Postsecondary Education Commission 2015; Colorado Commission 
on Higher Education 2015; Connecticut Board of Regents for Higher Education 2015; Kansas 
Board of Regents 2015; Minnesota Office of Higher Education 2015; Wyoming Community 
College Commission 2010). These indicator groups helped identify report card trends and key 
metrics used in this study. 

Performance Funding 

The linkage of performance reporting systems and funding is clearly stated in a representative 
sample state-based systems (Mississippi Institutions of Higher Learning 2015; Nevada System of 
Higher Education 2015; Pennsylvania State System of Higher Education 2011; Tennessee Higher 
Education Commission 2015; Utah Foundation 2014); in general, stakeholders want to link 
funding to performance in an effort to force positive, data-based change typically in areas of 
access, minority student success rates, financial efficiency and workforce needs (Dougherty, 
Natow, Hare, Jones & Vega 2011). However, the impact of performance funding in higher 
education is mixed; despite early positive reports of performance funding for facilitating 
standards-based implementation and accreditation processes, (Williams 2005), recent literature 
paints a more critical picture. Dougherty and Reddy (2011) literature review comparing studies 
on institutional-level funding had a common finding that awards amount were small, but 
effectiveness ranged from either low impact or service as an additional revenue fund, to serious, 
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substantial changes. A policy brief by the Columbia University Community College Research 
Center (2014) noted that while performance funding has led to institutional changes, it has not 
been linked to increased resource allocation nor increased retention and graduation rates. 
Dougherty, Natow, Hare, Jones, and Vega's (2011) literature review findings are even bleaker, 
with only half of U.S. states adopting performance funding models and more than half of those 
either abandoning the effort or substantially changing the model. It is reasonable to assume that 
reporting structures will continue to be in flux while the funding that supports them continues to be 
responsive to political and economic variables and trends. 

By comparing how institutions treat instructional objectives and learning outcomes, the presented 
study appears to offer a unique framework not often encountered in transparency literature; a 
skewed emphasis on the importance on learning outcomes is clearly noted in recent literature 
(Protz, 2015); in fact, Wittman-Price and Fasolka (2010) argue that this practice, while well- 
aligned for promoting quality improvement mask the need for change. This study's equal 
emphasis on objectives and outcomes allows for different expressions of educational paradigms 
to weigh into analytic comparison. 


METHOD 

Study design, population, and sampling procedure 

A cross-sectional content analysis of transparency reports offered by 82 colleges and universities 
in the United States was employed and categorized into: 17 Carnegie class research-emphasis 
public universities, 19 state colleges or universities, 22 community colleges, and 24 private 
colleges or universities; differences in the size of the four categories occurred to achieve a valid, 
representative sample size. Stratified sampling procedure was adopted in selecting the 
representative institutions in order to reflect the findings of Gordon and Fischer (2008) who noted 
a relationship between informational disclosure and institutional factors such as size and type. 
Data was collected by analyzing institutional research, institutional effectiveness, as well as home 
page entries hosted for the purposes of factual comparison. The selection of institutions 
dynamically influenced the coding tool in that every potential finding of a pilot group was 
considered for inclusion in the developing coding tool. This method of coding has been shown to 
enable researchers to examine datasets in a systematic fashion to determine focus (Weber 
1990). It also allows inferences to be made which can then be corroborated using other methods 
of data collection. Data was coded based on the title of represented data, so data represented as 
“Quick Facts” was coded as such. Data without a representative title was coded based on an 
analysis of data, so data of a more promotional variety, including rankings was coded as a 
promotional fact book. In cases were the same title was used for two different documents or web 
pages, the data was analyzed to determine correct placement, again using features such as 
rankings or performance measures to indicate placement. Since some data appeared in multiple 
locations, the accessibility of the data was considered the primary factor, so data listed in a 
prominent homepage position was coded over data that could only be obtained after multiple 
clicks. 

In regards to coding individual metrics, literature review findings indicated a good potential for 
disaggregation of metrics, however, with metric trends identified from the literature review, this 
study grouped metrics to their identification of goals, standards, and benchmarks for student 
success, objectives, criteria and measures for demonstrating institutional and unit (program) 
quality, identification of student-based factors, and measures of satisfaction as the primary 
grouping mechanisms. 


Data collection methods and research tools 
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The framework for transparency reporting used in this study was developed using as a hybrid of 
inputs including: recommendation from field literature investigated in the study literature review; 
standards and guidelines from: Association of Institutional Research and its affiliates, the 
American Educational Research Association, the American Council on Education, and the 
Council for Higher Education and its regional accrediting bodies; and data collected via a coding 
spreadsheet from the pilot institutions of study which included data collected from relevant 
reporting and transparency documents including: report cards, factbooks, quick facts, promotional 
factbooks, surveys, impact studies, campus and student profiles, and assessment data. Due to 
the focus of this study on student achievement, other transparency reports and metrics such as 
crime, fire, and environmental reports were found, but not included in this study. Since the focus 
of this study was transparency reporting, data that required a login or password was not included 
in the study. 

The placement of data was considered of primary importance due to eye-tracking studies 
suggesting that website information consumers prefer a limited number of information points 
(Djamasbi, Siegel, Skorinko, & Tullis 2011). Therefore, the first five entries of each transparency 
report were mined for inclusion in the coding tool. Since the pilot study resulted in the 
development of a robust coding tool, in cases where data subsets beyond the coding category 
were included, a determination of inclusion of a new category was met only if the resultant data 
could be considered a new aspect or perspective. When multiple datasets were provided, the 
latest dataset was studied. 

Data analysis 

Quantitative data on 65 indicators was input into SPSS® for statistical coding and analysis to 
obtain simple descriptive uni-variate statistics to draw inferences and comparisons of the 
representatives of each of the four (4) categories of studied institutions. During coding, the data 
was classified, sorted and tabulated for easier comparison and inference. Document analysis of 
the curricula was done using a framework that heavily anchored identified knowledge areas and 
matched it against the sample knowledge areas to establish best fit. 

Reliability and validity of data collected 

Internal validity was achieved through controlling internal and external influences i.e. proper 
orientation of research assistants; as noted by Erlandson, Harris, Skipper, and Allen (1993), this 
triangulation method lends credibility to the findings by incorporating multiple sources of data, 
methods, and investigators. Similarly, external validity was achieved by following correct, 
unbiased sampling procedures and using a representative sample size of at least 20%. In this 
study, data validity was ensured by collecting data from reliable sources of institutional office web 
pages, using basic questions as identified in the related literature review, and pre-testing the 
questionnaire for meanings. 


RESULTS 

Due to the variety of institutional reports, web pages, PDFs and fact sheets presented on web 
communication channels, this study aggregated the data presentation by type, with the following 
definitions: Report Cards offered some type of performance management; in addition, it was clear 
that some report cards where being used to provide transparency for evaluation bodies and 
justification of expenditures to stakeholders, due to findings of state and local economic impact or 
analysis of impact on workforce development. Factbooks also are used to address transparency, 
however, career guidance and promotional aims were also found in the sample as evidenced by 
degree completion, placement rates, and licensure pass rates. Quickbooks generally took on a 
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promotional tone, however, transparency of data and career guidance were also found in the 
sample, particularly for high-interest field areas, such as STEM fields. Promotional reports were 
typically found in general areas off the institutional home page or an admissions site. Surveys and 
student profiles would most often be listed separately under the sponsorship of Institutional 
Research or Institutional Effectiveness Offices. Of the 82 studied colleges and universities, only 
one institution, a private institution, did not offer any type of transparency or data reporting in any 
category. It should be noted that across institutions, there was a consistent finding of confusion 
about the use of headcount and enrollment, with a frequent finding of the terms used 
interchangeably. For this study, headcount serves as a single figure in a category, while 
enrollment indicates a breakout of a population into identifiable groups. 

At an average of 8.12 documents, state colleges and universities averaged the highest number of 
documents and webpages devoted to transparency reporting. An average of 7.28 documents or 
web pages provided transparency data at Carnegie class research institutions. 64.7% of 
Carnegie-class research institutions and state colleges and universities employed an Office of 
Institutional Research to disseminate findings, with 29.4% of research institutions deploying an 
institutional planning office to perform transparency functions. For community colleges, an 
average of 6.375 documents or web pages was devoted to reporting; private institutions offered 
an average of 4.75 documents of webpages. The largest group of community colleges, 43.75%, 
hosted reports by an institutional research office, 31.25% of institutions did not indicate data 
sponsorship, and 25% were hosted by an institutional effectiveness office. For private institutions, 
47% hosted reports by an institutional research office while 23.5% did not provide an indication of 
data sponsorship. 

Report Cards 

89.3% of studied institutions offered some type of institutional report card, performance 
indicators, score card, dashboard, or student achievement reporting for consumers. Results were 
evenly distributed among the research, community college and private institutions, with all but 
one of the state-level colleges and universities providing some type of internal assessment. 
26.4% of institutions employed an external site to disseminate all or part of their performance- 
related information, with 66.6% of those employing the Integrated Postsecondary Education Data 
System's (IPEDS) College Navigator® project. The Voluntary System of Accountability's (VSA) 
College Portrait® project was deployed by 27.7% of institutions employing an external link. The 
Student Achievement Measure® (SAM), sponsored by a higher education consortium, is linked to 
by the final significance category by 16.6% of link-employing institutions. 

Enrollment and admission trends were the most significant reporting categories for report cards of 
Carnegie-class research institutions. Economic impact on local areas, retention and graduation 
rates and aggregates, institutional rankings, and value-added analysis of improvement or 
comparison benchmarks rounded out the other categories of significance with 17.6% of 
institutions participating in each. Only enrollment trends at 23.5% and to a lesser extent, graduate 
student participation and value-added analysis of improvement or comparison benchmarks at 
17.6% were significant findings for state colleges and universities. For community colleges, 
enrollment trends, retention and\or persistence rate, and credit hours and\or course completion 
were equally reported at 37.5% of community colleges. Information on graduation or completion 
rates, including to a lesser extent, aggregates on cohorts, freshmen, and transfer students, saw 
the second highest reporting rate of 25%. Finally, number of degrees awarded, and to a lesser 
extent, category aggregates, along with reports on transfer students, and the economic impact on 
local areas served rounded out the final category of significance at 18.75%. 18.75% of community 
colleges presented data in the form of a dashboard indicating alignment to goal attainment. 
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For private institutions, graduation or completion rates were the most reported data items at 
29.4% of institutions. 23.5% of private institutions presented data related to placement rates. 
Retention rates, credit hours and\or course completion, and athletic participation were equally 
represented in the final tier of significant participation with 17.6% of institutions providing 
information. 

Factbooks 

76.4% of research level public institutions provide enrollment data in factbooks, followed by 47% 
providing admissions data such as number applied, admitted, enrolled as well as aggregate data 
for freshmen and transfer students. 35.2% of research-level factbooks provided headcount 
information. Similar percentages were found for the provision of state colleges and universities 
enrollment data in factbooks at 82.3%. Faculty and staff data (51%) and retention data (41.1%) 
were also significant findings for this category. For community colleges, enrollment (29.4%), 
student characteristics or profile (23.5%), and credit hours and aggregates such as class, 
schedule, instructional method and subject, tied with institutional information including history, 
logo, alma mater, president, organizational chart, type, calendar, mission and vison (17.6%) 
served as areas of significance in publicly posted factbooks. Participation is stronger for private 
institutions, with 35.2% including enrollment information, 29.4% each for headcount and faculty 
and staff information and aggregates, and 23.5% each for costs and financial aid information and 
number and type of academic programs. 

Fast Facks\Quick Facts\At a Glance 

Typically located off the institutional homepage's About Us menu, this subset of factual data is 
meant to be quickly absorbed by the consumer. Headcount, the leading indicator across 
institutional type for this category and enrollment data and aggregates were the most significant 
inclusion (53%) in fast facts for Carnegie-class research institutions. At 29.4%, institutional data 
including president, leadership, address, and location; and founding date as well as facilities, 
buildings, and grounds were slightly more significant than the categories of credit hours, 
accreditation information including faculty degrees, and student profiles or characteristics 
(23.5%). Headcount (41.1%); followed by admissions data and institutional information such as 
president, leadership, address, and location; and founding date (29.4%), and number and 
aggregates for degrees awarded (23.5%) make up the state colleges and university category for 
fast facts. For community colleges, headcount (53%) followed by credit hours and student profile 
or demographics (29.4% each) and cost (23.5%) are the most significant categories. Headcount, 
student\faculty ration or class size, and student profile or demographics (35.3%) were the most 
significant categories for private institutions, followed by enrollment, admissions data, and 
institutional information such as president, leadership, address, and location; and founding date 
(29.4%). Number of academic programs was the last significant category for this type of 
institution (23.5%). 

Promotional Facts 

Typically titled By the Numbers, Facts and Rankings, Did You Know? or Pride Points, two 
categories of institutions, research-level institutions and state colleges and universities had 
significant findings for promotional facts that are typically located in a prominent location of the 
main menu in either the About Us or targeted admission page. Institutional rankings (41.1%), 
followed by the number of institutional scholars, fellows, Nobel prizes, and\or important faculty, 
alumni, and programs tied with institutional headcount as significant for Carnegie-class research 
institutions. State colleges and universities only had significant findings for headcount (29.4%) 
and number of academic programs (23.5%) in this category. 
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Student Profiles or Demographics 

While several schools presented data in this area, community colleges had the most significant 
findings with 64.7% presenting gender data about students, 47% presenting ethnicity information, 
and 35.3% providing age and full\part time student status. 29.4% of research level institutions 
presented data concerning headcounts and degree-seeking student aggregates. 

Diversity reports 

Of school categories that presented data in this area, only Carnegie-class research institutions 
presented significant data in the form of 23.5% of institutions providing student enrollment and 
faculty and staff percentages for this category. 

Survey data 

For Carnegie-class research institutions, 35.3% of schools provided student satisfaction data, 
usually in the form of a National Survey of Student Engagement (NSSE) report. Senior or 
graduating students; faculty, staff, or employee; and alumni surveys and data, followed by 
graduate student and campus climate surveys were also significant findings for research 
institutions in this category. National Survey of Student Engagement (NSSE) reports (47%); 
followed by senior or graduating student surveys (35.3%); and faculty, staff, and employee and 
alumni survey and data (23.5%) rounds out the significant findings for this category. 23.5% of 
private institutions provided survey data regarding seniors or recent graduates. 


DISCUSSION 

While only a small number of institutions provided results in the form of a dashboard, field 
evidence exists that dashboard presentation of data may be poised for greater deployment since 
53.7% of a 467 Association of Institutional Research (AIR) member population admitted to 
managing at least one dashboard (Association of Institutional Research 2013). In fact, Reindle 
and Reyna (2011) list efficiency and effectiveness metrics including meeting workplace needs, 
student output relative to input, return on investment, and quality of student learning as the most 
important factors to boost productivity and the second most important type of metric for gauging 
performance and targeting improvement areas. Reyna (2010) included a category of “progress 
metrics” including remedial enrollment and success, credit accumulation, first-year success, as 
well as retention rates and course completion as important for policymakers and interventions 
requiring budgetary support. 

While the majority of institutions seem to be aware of the importance of reporting graduation and 
retention rates and placement or employment information, the Association of the Study of Higher 
Education (2013), coined a category of “ultimate student outcomes” to represent aggregate 
retention reports, remedial coursework and credit hour reporting, and college readiness 
dashboards. This study confirms that a growing number of institutions are either responding to 
performing funding policies to address equity or are following field recommendations (Jones 
2014; Offenstein, Moore & Shulock 2010; Offenstein & Shulock 2010) to provide transparency in 
order to increase understanding of student representation groups. While a significant number of 
institutions were found to present degree production, gradation and retention rates, high-demand 
degree production was rarely reported in this study's sample and excess credit production was 
not reported by any institutions; both of these measures were recommended by the Washington 
Higher Education Coordinating Board (2011). 
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The relatively high degree of financial reporting of incomes, revenues, costs, and expenses may 
be influenced by calls to use of performance metrics to justify institutional allocations (Reindle & 
Reyna 2011). However, while a significant number of institutions report on costs, revenues, and 
expenditures, few of the studied institutions are heeding the California Postsecondary Education 
Commission's (2008) call to include affordability as a report card item. In fact, an unexpected 
finding of this study is the low availability of progress or value-added metrics displayed in the 
studied sample. In a focus group on standards and expectations to improve accountability, Wilis, 
et al. (2010) confirmed the value that stakeholders place on progress metrics as that study 
sample rejected simplistic ratings in favor of metrics that recognized social conditions, and 
included expectations for closing achievement gaps. 

The studied sample presented several concerns about the presentation of the data itself. Due to 
recommendations about leaving a primary site to visit a secondary site, the 17.6% of institutions 
employing an external site to present their own outcomes in a primary manner are not heeding 
data dissemination best practices of web developers (Mackey 2002). Another concern is the use 
of the Common Data Set template to communicate primary information given that the template 
reads as a report from the institution to the agency, rather than to an audience. Users would need 
to understand the template framework to navigate to requested information, a standard not 
recognized by the Cooperative Education Data Collection and Reporting (CEDCAR) Standards 
Project Task Force (Bayless et al. 1991). 

The prominent finding of historical information in Factbooks and QuickFacts, such as founding 
date may serve as an introduction to facts rather than an abrupt transition. Confusing the whole 
point of transparency, the elevation of promotion-oriented facts, such as rankings may serve to 
obscure more important facts such as graduation and retention rates, some of which were only 
available via the Institutional Research page in the current sample. In fact, Locke (2014) argued 
that higher education's marketing of rankings has become a substitute for more authentic data. 

Due to the impact that federal financial aid has on enrollment and completion, Reimherr, Harmon, 
Strawn, and Choitz's (2013) recommendation for the inclusion of program and student outcome 
metrics, such as participation and progress in remedial education, completion of checkpoint 
courses, and credential program persistence and completion, incentive the serving of at-risk 
students. In this study, the reporting of the number of Pell grant recipients, and the establishment 
of diversity reports may support incentives for serving at-risk students. In fact, Reyna (2010) 
reported low-income and remedial students as the largest categories of students overlooked by 
federal aggregated data groups, which has implications for state and federal policies and 
spending on time-to-degree and remedial interventions. In a related area, Fabricant (2014) 
recommends the use of “austerity metrics” such as number of students enrolled part-time due to 
financial constraints and time-to-completion rates, in addition to tuition costs in order to better 
understand how the higher education cuts and tuition privatization has impacted middle and lower 
class families. Finally, the institutions of this study are not heeding field literature suggestions 
(Burke, Hedrick, Ouelette, & Thompson, 2008) for incorporating metrics related to individuals with 
disabilities in order to explore issues and data unique to this population and foster a more 
inclusive environment; in fact reporting on enrollment, retention or completion did not appear as a 
significant aggregate for any of the studied metrics. 

While not a significant finding, the number of institutions reporting transfer out rates and student 
progress in terms of achieved credit benchmarks seems to reflect recent recommendations 
(Jones 2014) to reconsider the utility of commonly used metrics in order to better quantify the 
increasingly common phenomena of attendance at multiple institutions. 


Recommended Taxonomy 
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Based on study findings, a description of recommended category findings of performance 
indicators is provided below: 

a) Student and participation indicators (number of enrolled students; credit hour load of 
students enrollment aggregates including sex, ethnicity, resident, age, education level); 

b) Staff and leadership indicators (number of employees; employee aggregates including 
percentage of faculty types, faculty degree qualifications; percentage change in number 
of administrators; salaries); 

c) Financial, cost, affordability indicators (operating income, revenue and expenses and 
aggregates; salaries and costs as a proportion of total expenditure; costs per relevant 
student groups; revenues from research; cost aggregates (per credit, cost to graduation) 
and affordability aggregates (by institution, compared to benchmarks); 

d) Research indicators (share of research student groups; revenues from research 
generated by students in postgraduate studies; revenues from research per researcher; 
research production and quality indicators (number of publications, national and 
international presentations); 

e) Program, quality, outcome, learning and success indicators (number of accreditations, 
number and quality of reviews; rankings, number of licensures and certifications; rate of 
progression of students; value added success, field test scores and percent pass rates); 

f) Access and diversity indicators (number of initiative or funding support for diversity 
initiatives, percent of income for family burden, percent of Pell-grant recipients, 
enrollment aggregates); 

g) Preparation and college readiness indicators (means for SAT\ACT scores, high school 
GPA, rank in high school, percent needing remediation, progression in remediation, 
number of ESL, parent educational level); 

h) Persistence indicators (retention and transfer out rates, cohort rates, indications of 
reasons for leaving); 

i) Completion indications (number of graduates and completers for 100% and 150%, 
aggregates by program); 

e) Workforce development\area impact\employment indicators (placement rates, graduates 
employed full-time, graduates who continue their regular studies, the average wages of 
graduates; impact on local and state economy, number of STEM and high need 
graduates, pipeline analysis). 


CONCLUSION 

Criteria emerging from study indicate that institutional transparency reports are increasingly 
serving a variety of purposes including transparency, career guidance, promotional, performance 
management, and evaluation and justification to stakeholders, yet, with the exception of 
promotional report cards, audience considerations do not to appear to be a consideration of many 
report cards with large volumes of data dumped without definitions or context provided for the 
reader; findings which go against the primary recommendation of field literature for consideration 
of audience in performance reporting (Wiggens,1994). Findings of significant measures indicate 
that institutions value the clear communication of achievement and progress over time, but may 
not be providing accurate descriptions of learning or achievement of identified learning objectives. 
Too often, results are not standard-referenced or benchmarked against meaningful comparison 
groups. Our study joins other field literature in calling for a weighing of evidence of existing 
performance-based funding policies; however, for states still considering this model, a 
consideration of Klein’s (2015) seven steps to develop a formula for performance-based funding 
due to its linkages of state commitment and funding priorities to implementation. In addition, 
states should keep Miao's (2012) study of best practices of performance-based funding in mind 
due to its considerations of different funding for different settings, model consideration of budget 
cuts and funding constraints, stop-loss provisions, and implementation transition period. 
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